

in urging direct measures of teaching quality says that, often defining and
operationalizing the qualities and behaviors that go into good teaching, samples
of teachers could be observed periodically to provide the needed data on overall
and special pedagogical attributes.

Other Study Designs.  Some writers suggest that NCES use several other types of
designs to collect data in addition to the survey and observational designs
previously reviewed.

The National Education Association (NEA) recommends that NCES consider expanding
its "program to include case studies, field studies, policy reviews, (and)
historical research." A number of other papers support the use of a case-study
approach.  Banner points out the importance of historical statistics in
establishing "the context for the interpretation of long-term change" and the
"perimeters of contemporary issues." Royston, writing for the U.S. Equal
Employment Opportunity Commission, who also supports the case study design,
suggests ethnographic studies "on how minority students 'move through1 the
system, with emphasis on barriers and 'tracking. "r

THE NECESSARY CONDITIONS FOR COMPARABILITY TO ASSESS DIFFERENCES AND CHANGES

Walberg concisely states:  "The value of statistical research depends on valid
comparisons.,.." Selden reviews three ways that data can be used in conducting
comparisons.  The data may be compared with a standard, with data from another
group, or with data from the past. All three forms of comparisons require
accurate data.  This section synthesizes comments relating to the comparability
of data.

The Range of Definitional Problems.  A phrase by Cooke, Ginsburg and Smith
captures the essence of the single issue raised most frequently relating to data
compatibility:  "Data can be reported incorrectly if definitions across
respondents are inconsistent...." They provide as examples the problems created
by the lack of consistent definitions in determining dropout rates, student
achievement, student victimization, class size, teacher salaries, and of course
enrollments.

The paper by The National Center for Research in Vocational Education also
provides examples of problems controlled by definitional inconsistenaies.  Their
examples include the lack of definitions for the following terms:  vocational
students, disadvantaged and handicapped students, and program completers or
leavers.

Berryman lists examples of inconsistent definitions in the collection of data
pertaining to teacher supply and demand and data on education revenues and
expenditures by categories.  She states that "data such as these are usually not
comparable across states because the definitional variables differ."

Barker, writing for the Rural Education Association (REA), suggests that the
"lack of a precise definition may be one reason rural education has received
little attention in recent years." For example, even the concept of "rural" is
defined differently by the Bureau of the Census and some of the States.

77rs allocate their time can be collected by
